Non-local adaptive loop filter processing

ABSTRACT

Aspects of the disclosure provide a method for non-local adaptive loop filtering. The method can include receiving reconstructed picture, dividing the picture into current patches, forming patch groups each including a current patch and a number of reference patches, determining a noise level for each of the patch groups, and denoising the patch groups with a non-local denoising technology. The determining a noise level for each of the patch groups can include calculating a pixel variance for a respective patch group, determining a pixel standard deviation (SD) of the respective patch group according to the calculated pixel variance by searching in a lookup table that indicates mapping relationship between patch group pixel SDs and patch group pixel variances, and calculating a noise level for the respective patch group based on a compression noise model that is a function of the pixel SD.

INCORPORATION BY REFERENCE

This present disclosure claims the benefit of U.S. ProvisionalApplication No. 62/472,000, “Method of Non-local Adaptive Loop Filterfor Video Coding” filed on Mar. 16, 2017, which is incorporated hereinby reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to video coding techniques.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent the work is described in thisbackground section, as well as aspects of the description that may nototherwise qualify as prior art at the time of filing, are neitherexpressly nor impliedly admitted as prior art against the presentdisclosure.

Block-based motion compensation, transform, and quantization are broadlyemployed for video compression to improve performance of videocommunication systems. However, due to coarse quantization and motioncompensation, compression noise can be introduced which causesartifacts, such as blocking, ringing, and blurring in reconstructedpictures. In-loop filters can be employed to reduce the compressionnoise, which can not only improve quality of decoded pictures, but alsoprovide high quality reference pictures for succeeding pictures to savecoding bits. A non-local adaptive loop filter is one type of suchin-loop filter.

SUMMARY

Aspects of the disclosure provide a method for non-local adaptive loopfiltering. The method can include receiving reconstructed video datacorresponding to a picture, dividing the picture into current patches,forming patch groups each including a current patch and a number ofreference patches that are similar to the current patch, determining anoise level for each of the patch groups, and denoising the patch groupswith a non-local denoising technology to create a filtered picture basedon the determined compression noise level of each of the patch groups.The determining a noise level for each of the patch groups can includecalculating a pixel variance of pixel values for a respective patchgroup based on either the pixels in the patch group or the pixels in thecurrent patch of the patch group, determining a pixel standard deviation(SD) of the respective patch group according to the calculated pixelvariance for the respective patch group by searching in a first lookuptable that indicates mapping relationship between patch group pixel SDsand patch group pixel variances, and calculating a noise level for therespective patch group based on a compression noise model that is afunction of the pixel SD of the respective patch group.

In one example, the first lookup table can include a first sequence ofpatch group pixel SDs, and a second sequence of patch group pixelvariances. An average value between square values of each pair of twosuccessive patch group SDs forms the second sequence of patch groupvariances. In one example, the searching in the first lookup table caninclude searching for a range defined by two successive average valuesin the second sequence that includes the pixel variance of therespective patch group, and determining a patch group SD in the firstsequence to be the pixel SD of the respective patch group. A squarevalue of the determined patch group SD is included in the same rangedefined by the two successive average values in the second sequence.

In one example, constructing of the first lookup table can includedetermining a range of the first sequence of patch group SDs,determining a precision of the first sequence of patch group SDs, anddetermining the first sequence of patch group SDs according to thedetermined range and precision.

In an embodiment, the first lookup table includes a first sequence ofpatch group SDs, and a second sequence of patch group variances. Each ofthe first sequence of patch group SDs is an SD of one of the secondsequence of patch group variances. The searching in the first lookuptable can include finding a patch group variance in the second sequencethat is equal to the pixel variance of the respective patch group, anddetermining one of the first sequence of patch group SDs correspondingto the found patch group variance of the second sequence to be the pixelSD of the respective patch group.

In various examples, the non-local denoising technology can be one of anon-local means (NLM) denoising technology, a block matching and 3Dfiltering (BM3D) denoising technology, or a low-rank approximation (LRA)denoising technology.

In one embodiment, the denoising the patch groups with the non-localdenoising technology to create the filtered picture can includecalculating an ASSE/Var value to be used for calculating a weightingfactor of a reference patch with respect to a current patch, ASSE beingan average of a sum of square errors between the reference patch and thecurrent patch, Var being a noise variance that is a square of a noiselevel of a patch group that includes the reference patch and the currentpatch. The calculating the ASSE/Var value can include right shifting theVar by N_(x) bits to locate K number of most significant bits (K-MSBs),K being a preconfigured number, determining a value of 2̂M/K-MSBs bysearching a second lookup table based on the located K-MSBs, M being apreconfigured number, right shifting the ASSE by N_(x) bits, multiplyingthe right-shifted ASSE with the determined value of 2̂M/K-MSBs to obtaina product, and right shifting the product by M bits to obtain theASSE/Var value.

In one example, the second lookup table indicates mapping relationshipbetween a sequence of values of 2̂M/K-MSBs and a sequence of K-MSBs eachhaving K number of bits.

Embodiments of the method can further include one of adding a roundingoffset to the Var before right shifting the Var by N_(x) bits, adding arounding offset to the ASSE before right shifting the ASSE by N_(x),bits, and adding a rounding offset to the product before right shiftingthe product by M bits.

In an embodiment, the denoising the patch groups with the non-localdenoising technology to create the filtered picture can includecalculating a normalized pixel value corresponding to an accumulatedpixel value X_(A) that includes multiple portions of pixel values, eachof the portions corresponding to an original pixel value of the dividedpicture multiplied by a gain. The calculating the normalized pixel valuecan include (a) right shifting a sum of the gains, S_(G), correspondingto the accumulated pixel value X_(A) by N_(x) bits to locate K number ofmost significant bits (K-MSBs), K being a preconfigured number, (b)determining a value of 2̂M/K-MSBs by searching a third lookup table basedon the located K-MSBs, M being a preconfigured number, (c) rightshifting the X_(A) by N_(x) bits, (d) multiplying the right-shiftedX_(A) with the determined value of 2̂M/K-MSBs to obtain a product, and(e) right shifting the product by M bits to obtain a first normalizedpixel value.

In one example, the third lookup table indicates mapping relationshipbetween a sequence of values of 2̂M/K-MSBs and a sequence of MSBs eachhaving K number of bits.

In an embodiment, the calculating the normalized pixel value can furtherinclude performing steps of (a)-(e) with a rounding offset operation toobtain a second normalized pixel value, and averaging the first andsecond normalized pixel values to obtain the calculated normalized pixelvalue. The rounding offset operation can include at least one of addinga rounding offset to the S_(G) before right shifting the S_(G) by N_(x)bits, adding a rounding offset to the X_(A) before right shifting theX_(A) by N_(x) bits, or adding a rounding offset to the product beforeright shifting the product by M bits.

Aspects of the disclosure provide a non-local adaptive loop filter. Thenon-local adaptive loop filter can include circuitry configured toreceive reconstructed video data corresponding to a picture, divide thepicture into current patches, form patch groups each including a currentpatch and a number of reference patches that are similar to the currentpatch, determine a noise level for each of the patch groups, and denoisethe patch groups with a non-local denoising technology to create afiltered picture based on the determined compression noise level of eachof the patch groups. The determining the noise level can includecalculating a pixel variance of pixel values for a respective patchgroup based on either the pixels in the patch group or the pixels in thecurrent patch of the patch group, determining a pixel standard deviation(SD) of the respective patch group according to the calculated pixelvariance for the respective patch group by searching in a first lookuptable that indicates mapping relationship between patch group pixel SDsand patch group pixel variances, and calculating a noise level for therespective patch group based on a compression noise model that is afunction of the pixel SD of the respective patch group.

Aspects of the disclosure provide a non-volatile computer readablemedium storing instructions implementing the method for non-localadaptive loop filtering.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of this disclosure that are proposed as exampleswill be described in detail with reference to the following figures,wherein like numerals reference like elements, and wherein:

FIG. 1 shows a video encoder according to an embodiment of thedisclosure;

FIG. 2 shows a video decoder according to an embodiment of thedisclosure;

FIG. 3 shows a process for denoising a reconstructed picture accordingto an embodiment of the disclosure;

FIG. 4 shows an exemplary process of denoising according to anembodiment of the disclosure;

FIG. 5 shows a first example patch group pixel standard deviation (SD)lookup table according to an embodiment of the disclosure;

FIG. 6 shows a second example patch group pixel SD lookup tableaccording to an embodiment of the disclosure;

FIG. 7 shows a process for calculating a term ASSE/Var using the thirddivision lookup table according to an embodiment of the disclosure;

FIG. 8 shows a process for calculating a term X_(A)/S_(G) according toan embodiment of the disclosure; and

FIG. 9 shows a process with rounding offset operations according to anembodiment of the disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

I. Non-Local Adaptive Loop Filters

FIG. 1 shows an encoder 100 according to an embodiment of thedisclosure. The encoder 100 can include a decoded picture buffer 110, aninter-intra prediction module 112, a first adder 114, a residue encoder116, an entropy encoder 118, a residue decoder 120, a second adder 122,and one or more in-loop filters, such as a deblocking filter (DF) 130, asample adaptive offset filter (SAO) 132, an adaptive loop filter (ALF)134, and a non-local adaptive loop filter (NL-ALF) 136. Those componentscan be coupled together as shown in FIG. 1.

The encoder 100 receives input video data 101 and performs a videocompression process to generate a bitstream 102 as an output. The inputvideo data 101 can include a sequence of pictures. Each picture caninclude one or more color components, such as a luma component or achroma component. The bitstream 102 can have a format compliant with avideo coding standard, such as the Advanced Video Coding (AVC)standards, High Efficiency Video Coding (HEVC) standards, and the like.

According to an aspect of the disclosure, the NL-ALF 136 can employnon-local denoising techniques to improve the performance of the encoder100. Specifically, in one example, the NL-ALF 136 can divide areconstructed picture into a plurality of patches (referred to ascurrent patches). For each current patch, the NL-ALF 136 searches forsimilar patches (referred to as reference patches) in the reconstructedpicture to form a patch group. Subsequently, the NL-ALF 136 can apply adenoising technology to each patch group to modify pixel values of oneor more patches in respective patch group to reduce compression noise inthose patches. The modified pixel values are returned to the picture toform a filtered picture.

In addition, when applying a denoising technology to process each patchgroup, the NL-ALF 136 can employ certain techniques to lowercomputational complexity and simplify respective processing operations.For example, square root or division operations can be transformed tolookup table searching, bit shifting, multiplication operations, and thelike. As a result, implementation cost of the NL-ALF 136 can be reduced,and performance of the NL-ALF 136 can be improved.

Further, according to an aspect of the disclosure, it cannot beguaranteed that a processed pixel in the filtered picture is better interms of noise level than a corresponding unfiltered pixel in thereconstructed picture. Accordingly, the NL-ALF 136 can adaptivelydetermine for different blocks (regions) in the picture whether a blockwould adopt the processed pixel values or retain the unfiltered pixelvalues of the reconstructed video data. An on/off control flag can beemployed for signaling the adaptive adoption of the processed pixelvalues in a respective block.

In FIG. 1, the decoded picture buffer 110 stores reference pictures formotion estimation and motion compensation performed at the inter-intraprediction module 112. The inter-intra prediction module 112 performsinter picture prediction or intra picture prediction to determine aprediction (or, a predictor) for a block of a current picture during thevideo compression process. A current picture refers to a picture in theinput video data 101 that is being processed in the inter-intraprediction module 112. The current picture can be divided into multipleblocks with a same or different size for the inter or intra predictionoperations.

In one example, the inter-intra prediction module 112 processes a blockusing either inter picture coding techniques or intra picture codingtechniques. Accordingly, a block encoded using inter picture coding isreferred to as an inter coded block, while a block encoded using intrapicture coding is referred to as an intra coded block. The inter picturecoding techniques use the reference pictures to obtain a prediction of acurrently being processed block (referred to as a current block). Forexample, when encoding a current block with inter picture codingtechniques, motion estimation can be performed to search for a matchedregion in the reference pictures. The matched region is used as aprediction of the current block. In contrast, the intra picture codingtechniques employ neighboring pixels of a current block to generate aprediction of the current block. The neighboring pixels and the currentblock are within a same picture. The predictions of blocks are providedto the first and second adders 114 and 122.

The first adder 114 receives a prediction of a block from theinter-intra prediction module 112 and original pixels of the block fromthe input video data 101. The adder 114 then subtracts the predictionfrom the original pixel values of the block to obtain a residue of theblock. The residue of the block is transmitted to the residue encoder116.

The residue encoder 116 receives residues of blocks, and compresses theresidues to generate compressed residues. For example, the residueencoder 116 may first apply a transform, such as a discrete cosinetransform (DCT), wavelet transform, and the like, to received residuescorresponding to a transform block and generate transform coefficientsof the transform block. Partition of a picture into transform blocks canbe the same as or different from partition of the picture intoprediction blocks for inter-intra prediction processing.

Subsequently, the residue encoder 116 can quantize the coefficients tocompress the residues. The quantization can be controlled with aquantization parameter (QP). A QP indicates a step size for associatingthe transform coefficients with a finite set of steps. A larger QP valuerepresents bigger steps that crudely approximate the transform such thatmost of signals in the transform block can be captured by fewercoefficients. In contrast, a smaller QP value can more accuratelyapproximate the transform, however, at a cost of increased bit numberfor encoding the residues. Accordingly, a larger QP can induce moredistortion or compression noise into a picture resulted from the videocompression process. The compressed residues (quantized transformcoefficients) are transmitted to the residue decoder 120 and the entropyencoder 118.

The residue decoder 120 receives the compressed residues and performs aninverse process of the quantization and transformation operationsperformed at the residue encoder 116 to reconstruct residues of atransform block. Due to the quantization operation, the reconstructedresidues are similar to the original resides generated from the adder114 but typically are not the same as the original version.

The second adder 122 receives predictions of blocks from the inter-intraprediction module 112 and reconstructed residues of transform blocksfrom the residue decoder 120. The second adder 122 subsequently combinesthe reconstructed residues with the received predictions correspondingto a same region in the picture to generate reconstructed video data.The reconstructed video data can then, for example, be transferred tothe DF 130.

In one example, the DF 130 applies a set of low-pass filters to blockboundaries to reduce blocking artifacts. The filters can be appliedbased on characteristics of reconstructed samples on both sides of blockboundaries in a reconstructed picture as well as prediction parameters(coding modes or motion vectors) determined at the inter-intraprediction module 112. The deblocked reconstructed video data can thenbe provided to the SAO 132. In one example, the SAO 132 receives thedeblocked reconstructed video data and categorizes pixels in thereconstructed video data into groups. The SAO 132 can then determine anintensity shift (offset value) for each group to compensate intensityshifts of each group. The shifted reconstructed video data can then beprovided from the SAO 132 to the ALF 134. In one example, the ALF 134 isconfigured to apply a filter to reconstructed video data to reducecoding artifacts in the temporal domain. For example, the ALF 134selects a filter from a set of filter candidates and applies the electedfilter to a region of the reconstructed video data. In addition, the ALF134 can be selectively turned on or off for each block of thereconstructed video data. The processed reconstructed video data canthen be transmitted to the NL-ALF 136.

As described above, the NL-ALF 136 can process the receivedreconstructed video data using non-local denoising techniques to reducecompression noise in the reconstructed video data. In addition, theNL-ALF 136 can determine whether the non-local adaptive filtering isapplied for a block in a denoised picture. For example, the NL-ALF 136processes the received reconstructed video data and generates filteredvideo data. The NL-ALF 136 can then compare a filtered block of thefiltered video data with a corresponding block of the receivedreconstructed video data to determine whether a distortion of thefiltered block with respect to an original picture has been improved.When the distortion of the filtered block is improved, the pixel valuesof this filtered block can be adopted for forming the denoised picture.Otherwise, the pixel values of the corresponding block of the receivedreconstructed video data are adopted in the denoised picture.Accordingly, the denoised picture can be constructed based on thedecision of whether to adopt filtered pixel values for a respectiveblock in the denoised picture. The denoised picture can then be storedto the decoded picture buffer 110.

An on/off control flag can be employed to signal the above decision forthe respective block to a decoder such that the decoder can process theblock in the same way. As shown in FIG. 1, on/off control flags 103indicating whether non-local adaptive loop filtering is applied torespective blocks are transmitted to the entropy encoder 118 in oneexample.

The entropy encoder 118 receives the compressed residues from theresidue encoder 116 and on/off control flags 103 from the NL-ALF 136.The entropy encoder 118 may also receive other parameters and/or controlinformation, such as intra prediction mode information, motioninformation, quantization parameters, and the like. The entropy encoder118 encodes the received parameters or other information to form thebitstream 102. The bitstream 102 including data in a compressed formatcan be transmitted to a decoder via a communication network, ortransmitted to a storage device (e.g., a non-transitorycomputer-readable medium) where video data carried by the bitstream 102can be stored.

FIG. 2 shows a decoder 200 according to an embodiment of the disclosure.The decoder 200 includes an entropy decoder 218, a residue decoder 220,a decoded picture buffer 210, an inter-intra prediction module 212, anadder 222, and one or more in-loop filters, such as a DF 230, an SAO232, an ALF 234, and a NL-ALF 236. Those components are coupled togetheras shown in FIG. 2. In one example, the decoder 200 receives a bitstream201 generated by an encoder, such as the bitstream 102 generated by theencoder 100, and performs a decompression process to generate outputvideo data 202. The output video data 202 can include a sequence ofpictures that can be displayed, for example, on a display device, suchas a monitor, a touch screen, and the like.

Similar to the encoder 100 in FIG. 1 example, the decoder 200 employsthe NL-ALF 236, which has a similar function as the NL-ALF 136, todenoise a reconstructed picture to obtain a filtered picture. Forexample, when applying a denoising technology to process each patchgroup, the NL-ALF 236 can employ certain techniques to lowercomputational complexity and simplify respective processing operations.As a result, implementation cost of the NL-ALF 236 can be reduced, andperformance of the NL-ALF 236 can be improved. However, different fromthe NL-ALF 136 in FIG. 1 example, the NL-ALF 236 receives on/off controlflags 203 from the bitstream 201, and accordingly determines whichblocks of pixel values in the filtered picture are to be included orexcluded in a denoised picture. For example, when a control flag 203 ofa block is in a state of on, filtered pixel values of the block in thefiltered picture are adopted to the corresponding block of the denoisedpicture, while when a control flag 203 of a block is in a state of off,pixel values of the block of in the reconstructed picture are adopted.

The entropy decoder 218 receives the bitstream 201 and performs adecoding process which is an inverse process of the encoding processperformed by the entropy encoder 118 in FIG. 1 example. As a result,compressed residues, prediction parameters, on/off control flags 203,and the like, are obtained. The compressed resides are provided to theresidue decoder 220, and the prediction parameters are provided to theinter-intra prediction module 212. The inter-intra prediction module 212generates predictions of blocks of a picture based on the receivedprediction parameters, and provides the predictions to the adder 222.The decoded picture buffer 210 stores reference pictures useful formotion compensation performed at the inter-intra prediction module 212.The reference pictures, for example, can be received from the NL-ALF236. In addition, reference pictures are obtained from the decodedpicture buffer 210 and included in the picture video data 202 fordisplaying to a display device.

The residue decoder 220, the adder 222, the DF 230, the SAO 232, and theALF 234 are similar to the residue decoder 120, the second adder 122,the DF 130, the SAO 132, and the ALF 134 in terms of functions andstructures. Description of those components is omitted.

The employment of a non-local adaptive loop filter, such as the NL-ALFs136 and 236, in a decoder or encoder reduces a noise level inreconstructed video data, resulting in high quality output pictures. Inaddition, when those high quality pictures are used as referencepictures for encoding succeeding pictures, bit rate for transmission ofthe compressed pictures can be decreased. Therefore, denoisingtechniques disclosed herein for improving performance of a NL-ALF canimprove performance and capability of a decoder or encoder whichincludes the NL-ALF.

While the FIG. 1 and FIG. 2 examples show a series of filters 130, 132,and 134, or 230, 232, and 234, that are included in the encoder 100 ordecoder 200, it should be understood that none or fewer of such filterscan be included in an encoder or decoder in other embodiments. Inaddition, the position of the NL-ALF 136 or 236 with respect to otherfilters can be different from what is shown in the FIG. 1 or FIG. 2examples. For example, the NL-ALF 136 can be arranged in front of otherfilters so that it is directly coupled to the adder 122, or at the endof the series of filters, or among the series of filters.

In various embodiments, the NL-ALFs 136 and 236 can be implemented withhardware, software, or combination thereof. For example, the NL-ALF 136or 236 can be implemented with one or more integrated circuits (ICs),such as an application specific integrated circuit (ASIC), fieldprogrammable gate array (FPGA), and the like. For another example, theNL-ALF 136 or 236 can be implemented as software or firmware includinginstructions stored in a computer readable non-volatile storage media.The instructions, when executed by a processing circuit, causing theprocessing circuit to perform functions of the NL-ALF 136 or 236.

It is noted that the NL-ALFs 136 and 236 implementing the denoisingtechniques disclosed herein can be included in other decoders orencoders that may have similar or different structures from what isshown in FIG. 1 or FIG. 2. In addition, the encoder 100 and decoder 200can be included in a same device, or separate devices in variousexamples.

II. Non-Local Denoising Technologies

FIG. 3 shows a process 300 for denoising a reconstructed pictureaccording to an embodiment of the disclosure. The process 300 can beperformed at the NL-ALF 136 or 236 in the FIG. 1 or FIG. 2 example, andthe FIG. 1 example is used to explain the process 300.

The process 300 starts from S301 and proceeds to S310. At S310,reconstructed video data is received at the NL-ALF 136. For example, thesecond adder 122 receives predictions from the inter-intra predictionmodule 122, and residues from the residue decoder 120, and combines thepredictions and residues to generate reconstructed video data. Invarious embodiments of the process 300, the reconstructed video data cancorrespond to a picture, a frame, a slice of a picture, or a predefinedregion of a picture. Accordingly, a picture or a reconstructed picturecorresponding to the reconstructed video data, can refer to a picture, aframe, a slice of a picture, or a predefined region of a picture, andthe like, in this disclosure. In addition, a filtered picture, or adenoised picture resultant from a picture corresponding to reconstructedvideo data can accordingly refer to a picture, a frame, a slice of apicture, or a predefined region of a picture, and the like in thisdisclosure. Depending on a number of in-loop filters employed and aposition of the NL-ALF 136 among those in-loop filters, thereconstructed video data can correspond to reconstructed video datagenerated from the residue decoder 120, or filtered reconstructed videodata generated from a filter adjacent and previous to the NL-ALF 136.

At S320, the received reconstructed video data (corresponding to thereconstructed picture) is divided into a plurality of patches. In oneexample, the patches have a same size and shape, and are non-overlappedwith each other. In other examples, the patches may have different sizesor shapes. In further examples, the patches may overlap with each other.Each patch includes a plurality of pixels, and is referred to as acurrent patch.

At S330, for each current patch, a plurality of similar patches(referred to as reference patches) are found in the reconstructedpicture, or in some examples, in a search window within thereconstructed picture. In one example, each reference patch has asimilar shape and size as the respective current patch. In addition, thereference patches can overlap each other, and the reference patches andthe respective current patch can overlap each other. Each current patchand the respective similar reference patches form a patch group. In oneexample, for each current patch, the K most similar patches are found,and the current patch and the respective K most similar patches form apatch group including K+1 patches. K indicates a number of similarpatches corresponding to a current patch, and K can have differentvalues for different current patches or patch groups. In one example,similar matches found for a respective current patch are not the mostsimilar reference patches but reference patches having a similaritymeasure above a threshold. The similarity measure can be obtained basedon a similarity metric. In one example, patch groups are formed for aportion of the current patches instead of for each current patch.

In various embodiments, various search methods can be used to search forthe K most similar reference patches for a respective current patch. Inaddition, in various embodiments, various similarity metrics can be usedto measure the similarity between a current patch and a reference patch.For example, the similarity metric can be a sum of absolute differences(SAD), or a sum of square differences (SSD), between corresponding pixelvalues in a current patch and a corresponding reference patch. Foranother example, pixel values in a current patch and a correspondingreference patch can be arranged as two vectors, and a L2 norm distancebetween those two vectors can be used as a similarity metric.

At S340, a denoising technology is applied to each patch group to modifypixel values of one or more patches in the respective patch group.Particularly, when applying the denoising technology to process a patchgroup, the NL-ALF 136 can employ certain techniques to lowercomputational complexity and simplify respective processing operations.For example, square root or division operations can be transformed tolookup table searching, bit shifting, and/or multiplication operations.

The modified pixel values of different patches either belonging to asame patch group or different patch groups are then aggregated to form afiltered picture, for example, by operations of weighted sum. In variousembodiments, various denoising technologies can be employed to processthe patch groups. For example, the denoising technology can be anon-local means (NLM) denoising technology, a block matching and 3Dfiltering (BM3D) denoising technology, or a low-rank approximation (LRA)denoising technology. However, the denoising technology applicable forprocessing the patch groups are not limited to the NLM, BM3D, or LRAtechnologies.

At S350, on/off control flags associated with a control block in adenoised picture are determined in a manner as described above. Theon/off control flags indicate whether the control block adopts filteredpixel values in the filtered picture or pixel values of thereconstructed picture corresponding to the received reconstructed videodata. The on/off control flags can be signaled from an encoder to adecoder. That is, the encoder determines the values of the on/offcontrol flags and incorporates the on/off control flags with thedetermined values in the bitstream, so that the decoder parses theon/off control flags from the bitstream to determine the values of theon/off control flags. In various embodiments, control blocks for controlof whether to adopt filtered pixel values resultant from the denoisingoperations at S340 can be defined or partitioned in various ways. Forexample, a partition of the control blocks can be consistent with apartition of coding units defined in the HEVC standard. Or, a partitionof the control blocks can be consistent with a block partition used in afilter, such as the DF 130, the SAO 132, or the ALF 134, for purpose ofcontrol a filtering operation. Alternatively, the control blockpartition can be determined according to noise characteristics ofdifferent regions in a picture. In various examples, the control blockpartition information can be signaled from an encoder to a decoder,derived at a decoder, or predefined as a default configuration at bothan encoder and a decoder.

At S360, the denoised picture are constructed based on the on/offcontrol flag decisions. For control blocks associated with an on flag,filtered pixel values resultant from the denoising operation as S340 areadopted for the respective control blocks, while for control blocksassociated with an off flag, pixel values of the received reconstructedpicture are adopted for the respective control blocks. Subsequently, areference picture can be generated based on the denoised picture. Forexample, the denoised picture can be stored to the decoded picturebuffer 110 to be used as a reference picture. Alternatively, dependingon the position of the NL-ALF 136 among other in-loop filters, thedenoised picture may first be processed by other in-loop filters beforestoring into the decoded picture buffer 110. The process 300 proceeds toS399, and terminates at S399.

FIG. 4 shows an exemplary process 400 of denoising according to anembodiment of the disclosure. The denoising in the process 400 employs adenoising technology that is one of the various denoising technologiesemployed at S340 in the process 300. Accordingly, the process 400 can beperformed in place of S340 in the process 300 to obtain a denoisedpicture. Similarly, for other denoising technologies employed at S340, aprocess corresponding to the respective denoising technology can beperformed in place of S340. The NL-ALF 136 is used as an example toexplain the process 400.

The process 400 starts from S401 and proceeds to S410. Before initiationof the process 400, patch groups corresponding to reconstructed videodata received at the NL-ALF 136 can have already been formed, forexample, by performing the steps of S310-S330 in the process 300. Thereceived reconstructed video data can correspond to a reconstructedpicture or a portion of the reconstructed picture (such as a slice ofthe reconstructed picture). Assume the received reconstructed video datacorresponds to a reconstructed picture below. The steps of S410-S440 areiterated for each patch group of the reconstructed picture. During theiteration, a patch group being processed in each round of the iterationis referred to as a current patch group.

At S410, weighting factors for each reference patch in a current patchgroup are calculated. The weighting factor of a reference patch can becalculated based on a similarity between the reference patch and arespective current patch in the current reference patch group. The morea reference patch is similar to the current patch, the larger theweighting factor of the reference patch. In one example, the weightingfactors are determined using the following expression,

w _(i,j) =e ^(−(ASSE/Var)).

In the above expression, i and j are patch indexes, w_(i,j) represents aweighting factor of a reference patch j with respect to the respectivecurrent patch i; ASSE represents an average of a sum of square errorsbetween corresponding pixel values in the patches i and j, and indicatesthe similarity degree between the patches i and j; Var can be defined asa variance of compression noise in the current patch group or a varianceof compression noise in the current patch of the current patch group,and Var can be referred to as a noise variance. Therefore, the noisevariance Var can be equal to a square of a standard deviation (SD) ofcompression noise of the current patch group when the definition of thenoise variance Var is the variance of compression noise in the currentpatch group; or the noise variance Var can be equal to a square of astandard deviation (SD) of compression noise of the current patch whenthe definition of the noise variance Var is the variance of compressionnoise in the current patch of the current patch group. An SD ofcompression noise of a current patch group or the current patch can befurther referred to as a noise SD of the respective patch group. A noiseSD of a patch group indicates a noise level of the respective patchgroup. Thus, a noise SD of a patch group can be referred to as a noiselevel of the respective patch group. In addition, the noise variance Varcan indicate strength of the filtering operation. The higher thecompression noise level, the higher the Var, and the larger thecorresponding weighting factors of the current patch group.

In the above example, the weighting factors of each reference patch arederived based on the expression, e^(−(ASSE/Var)). In order to reducecomputational cost of calculation a weighting factor of a referencepatch, piece-wise linear interpolation is used to approximate the valueof the expression, e^(−(ASSE/Var)) in one example. For example, thevalues of (ASSE/Var) at 6 control points are {0, 0.5, 1.0, 2.0, 4.0,6.0} and the corresponding weighting factors are {1.0000, 0.6065,0.3679, 0.1353, 0.0183, 0.0025}. Assuming a current value of (ASSE/Var),V_(C), has been obtained, two closest control points to the currentvalue V_(C) can be used to perform a linear interpolation (orextrapolation) to generate a corresponding weighting factor of(ASSE/Var).

At S420, pixel values in the current patch are accumulated. In oneexample, the accumulation of the current patch pixel values is performedin a way that weighted pixel values of corresponding pixels in eachreference patch are aggregated to corresponding pixel values of thecurrent patch based on the respective weighting factors of eachreference patch. In one example, the accumulation is performed accordingto the following expression,

x _(Ai)(p)=x _(i)(p)+Σ_(j=1) ^(K) w _(i,j) ·y _(j)(p),

where p is a pixel index, x_(Ai)(p) represents an aggregated pixel valueof the pixel, p, in the current patch i resultant from the aggregation,x_(i)(p) represents an original pixel value of the pixel, p, in thecurrent patch i before the aggregation, y_(j)(p) represents an originalpixel value of the pixel, p, in a reference patch j.

At S430, pixel values in each reference patch of the current patch groupare accumulated. In one example, the accumulation of pixel values of areference patch is performed in a way that a weighted pixel value of acorresponding pixel in the current patch are added to a correspondingpixel value of the reference patch based on the weighting factors of thereference patch. In a first example, the accumulation is performedaccording to the following expression,

y _(Aj)(p)=w _(i,j) ·x _(Ai)(p),

where p is a pixel index, y_(Aj)(p) represents an aggregated pixel valueof the pixel, p, in a reference patch j resultant from the aggregation,x_(Ai)(p) represents the aggregated pixel value of the pixel, p, in thecurrent patch i resultant from the aggregation at S420. In a secondexample, x_(i)(p) in the current patch i is used in the aboveaccumulation instead of x_(Ai)(p).

At S440, the accumulated pixel values of the current patch and thereference patches resultant from S420 and S430 are accumulated tocorresponding pixels in a picture, referred to as an accumulationpicture. As the reference patches and the current patch in the currentpatch group can overlap each other, a pixel in the accumulation picturemay receive pixel values from multiple patches, either the current patchor one or more reference patches. As the steps S410-S440 are performedfor each patch group, at S440, accumulated pixel values of therespective current or reference patches of each round of the iterationcan be accumulated to the accumulation picture. Accordingly, a pixel inthe accumulation picture can receive accumulated pixel values from oneor multiple patch groups. As an example, accumulated pixel values,x_(A)(p), in the accumulation picture can be rewritten as

x _(A)(p)=x _(i)(p)+Σ_(i=1) ^(m)Σ_(j=1) ^(K) w _(i,j) ·y _(j)(p),

where i is the index of patch groups in one picture, and there are totalm patch groups in one picture.

At S450, accumulated pixel values in the accumulation picture arenormalized to obtain a filtered picture. As a result of S440, anaccumulated pixel in the final accumulation picture can include multipleportions of pixel values, and each of the portions corresponds to anoriginal pixel value of either a current patch or a reference patchmultiplied by a gain. Each gain corresponds to a weighting factor, or aproduct of one or more weighting factors. Accordingly, an accumulatedpixel in the accumulation picture can be divided by a sum of those gainsto obtain a normalized pixel value. Those normalized pixel values formthe filtered picture. The process 400 proceeds to S499 and terminates atS499.

As an example, after performance of the process 400, the steps of S350and S360 in process 300 can be performed to the filtered picture todetermine the on/off control flag and construct a denoised picture.

III. Compression Nose Models

In the denoising technology presented in the process 400, the noisevariance of compression noise in a current patch group, Var, is used forcalculating the respective weighting factors, as shown in theexpression, w_(i,j)=e^(−(ASSE/Var)). According to an aspect of thedisclosure, the Var can be derived based on a compression noise model.In addition to the denoising technology presented in the process 400, acompression noise model can also be used in other denoising technologiesfor deriving a variance of compression noise in a patch group, such asthe denoising technologies applied at S340 of the process 300 asdescribed above. For example, a compression noise model can be used toderive a threshold for soft-thresholding or hard thresholding in theblock matching and 3D filtering (BM3D) denoising technology or thelow-ran approximation (LRA) denoising technology. Examples ofcompression noise models and usage of compression noise models aredescribed below.

Generally, a compression noise model describes relationship between acompression noise level in a patch group and factors that affect thecompression noise level. The factors can include characteristics ofcontent of the patch group, coding types associated with the patch group(for example, coded by intra prediction or inter prediction), andtransform quantization parameters (QPs) associated with the patch group.A compression noise model can be determined based on experiments.Subsequently, compression noise levels of patch groups can be estimatedbased on the determined compression noise model during a non-localdenoising process for processing a reconstructed picture. In variousexamples, compression noise models can take different forms, may havedifferent accuracy, and may be formulated differently for differentapplications.

In one example, a compression noise model used for various denoisingtechnologies is formulated with a polynomial function. For example, thecompression noise model takes the following form,

y=ax ³ +bx ² +cx+d,

where y represents a noise SD of compression noise of a patch group,which indicates a compression noise level of the patch group and asquare of which equals the noise Var of the patch group; x represents aSD of pixel values of the current patch group or the current patch(referred to as a pixel SD) and indicates a characteristic of content ofthe respective patch group; and the parameters (coefficients), a, b, c,and d, referred to as model parameters, are constants. As an example,during a usage of the compression noise model, the constants, a, b, c,and d, can first be selected from multiple sets of model parametersbased on coding type or QP associated with the respective patch group.Then, a pixel SD of pixel values of the current patch group or thecurrent patch, x, can be calculated. Finally, based on the obtainedmodel parameters, a, b, c, and d, and the pixel SD of pixel values, x, anoise SD of compression noise of a patch group can be calculated.

In one example, the candidate sets of model parameters of thecompression noise model are determined by conducting an experimentalprocess. For example, common sequences of test pictures can be used inmultiple coding processes at an encoder, such as the encoder 100. Foreach coding process, a set of picture coding type and transform QP canbe configured for a sequence of pictures. Corresponding to each set ofcoding type and transform QP, compression noise and pixel valuescorresponding to different patch groups in a picture can be obtained.Accordingly, multiple data points of SDs of compression noise and pixelvalues corresponding to each patch group can be obtained. Based on themultiple data points, a set of model parameters (coefficients) of thecompression noise model corresponding to the set of coding type andtransform QP can be determined by a regression analysis to the datapoints. As a result of the above experimental process, multiple sets ofmodel parameters of the compression noise model can be obtained eachcorresponding to a different set of coding type and transform QP. Table1 below show an example of candidate sets of model parameters. As shown,a candidate set of model parameters is obtained for each pair of codingtype and QP.

TABLE 1 Candidate Sets of Model Parameters QP 1 QP 2 QP 3 Coding Type I{a, b, c, d}₁ {a, b, c, d}₂ {a, b, c, d}₃ Coding Type II {a, b, c, d}₄{a, b, c, d}₅ {a, b, c, d}₆

In a conventional denoising technology, a power function is used forformulating a compression noise model. In one such example, thecompression noise model takes the form of a power function, y=ax^(b),where y represents a SD of compression noise of a patch group, xrepresents a SD of pixel values of the patch group, and a and b aremodel parameters to be selected. According to an aspect of thedisclosure, usage of a polynomial function, instead of a power function,to formulate a compression noise model can reduce computation complexityrelated with the compression noise model.

In one example, clipping operations are applied to the polynomialcompression noise model in order to guarantee a monotonic increaseproperty of the polynomial function. For example, a clipping operationcan be added to a value of a pixel SD of pixel values of a respectivepatch group, or a value of a noise SD of compression noise of therespective patch group resultant from the polynomial function.

In a conventional denoising technology, selection of model parameters ofa compression noise model is based on picture or slice levelinformation. For example, patch groups formed at steps of S310-S330 areincluded in a picture or a slice, and accordingly coding types and QPsdefined at a picture level or a slice level are utilized to select modelparameters of the compression noise model for each patch group withoutdistinguishing different patch groups. However, according to an aspectof the disclosure, compression noise level varies for different regionsin a picture according to prediction types, QPs, and predictions ofrespective regions. Accordingly, different set of model parameters of acompression noise model can be selected for patch groups distributed atdifferent locations within a picture. As a result, noise variances ornoise SDs of compression noise of different patch groups can be derivedmore accurately.

Accordingly, in one example, when selecting model parameters of acompression noise model, coding unit level information is utilized. Acoding unit of an encoded picture can include coding data of collocatedblocks of multiple color components (such as a luma component, twochroma components). For each color component, the coding unit caninclude one or more coding blocks and one or more transform blocks. Asan example, a usage of coding unit is specified in the HEVC standard. Inone example, coding unit level information includes a prediction typeand a transform QP that are applied to respective coding blocks ortransform blocks, respectively, in the respective coding unit.

In one example, a reconstructed picture or slice includes a plurality ofcoding units, and coding unit level information including a predictiontype and a QP is associated with each respective coding unit. Theprediction type information can be used for prediction operations, forexample, at the inter-intra prediction module 112 in FIG. 1 and the QPinformation can be used for quantization operations, for example, at theresidue encoder 116. When applying a denoising technology to a currentpatch group, for example, at S340 in process 300, and calculating a Varof compression noise of the current patch group for obtaining weightingfactors, coding unit level information can be used to determine modelparameters of a compression noise model. For example, a coding unit hasa block (of one color component) that overlaps the current patch of thecurrent patch group (of the same color component) (for example, atop-left pixel of the current patch is within the block of the codingunit), and a prediction type and QP of this coding unit are then used toselect a set of model parameters of the compression noise model for thecurrent patch group, for example, from the Table 1.

In addition, in the above example, for scenarios that the coding unit iscoded with inter prediction, a state of residues corresponding to thecoding unit is also considered for selection of the compression noisemodel parameters. For example, the QP of the coding unit is used forselection of the model parameters when there is a non-zero residue inthe coding unit resultant from a prediction operation, while an averagebetween the QP of the coding unit and one or more QPs of one or morepredictions of coding blocks of the corresponding coding unit is used asthe QP for selection of the model parameters when there is no residue inthe coding unit. In one example, a QP of a prediction corresponding to acoding block in a coding unit is a QP of a coding unit that overlaps theprediction in a reference picture including the prediction. Or inanother example, a QP of a prediction corresponding to a coding block ina coding unit is a QP in the slice header of a reference picture.

IV. Techniques for Reducing Computational Complexities

When applying a denoising technology to process each patch group, theNL-ALF 136 or 236 can employ certain techniques to lower computationalcomplexity and simplify respective processing operations. For example,square root or division operations can be transformed to lookup tablesearching, bit shifting, multiplication operations, and the like. As aresult, implementation cost of the NL-ALF 136 or 236 can be reduced, andperformance of the NL-ALF 136 or 236 can be improved.

FIG. 5 shows a first example patch group pixel SD lookup table 500according to an embodiment of the disclosure. As described above, whenapplying a non-local denoising technology to process a patch group, anoise level of the patch group can be estimated with a compression noisemodel that is a function of a pixel SD of the respective patch group. Toobtain the pixel SD of the patch group, a pixel variance for the patchgroup can first be calculated based on the pixels in the patch group orthe pixels in the current patch of the patch group. Subsequently, asquare root operation can be performed to the pixel variance to obtainthe pixel SD. For example, the pixel variance for the patch group can becalculated with the following expression,

${{{Pixel}\mspace{14mu} {varence}} = {\frac{1}{n}{\sum\limits_{k = 1}^{n}( {x_{k} - u} )^{2}}}},$

where n represents a number of pixels in a current patch and referencepatches in the patch group, x represents a pixel value of the patchgroup, and u represents an average pixel value of in the patch group.Accordingly, the pixel SD of the patch group can be calculated with thefollowing expression,

${{Pixel}\mspace{14mu} {SD}} = {\sqrt{{Pixel}\mspace{14mu} {varence}} = {\sqrt{\frac{1}{n}{\sum\limits_{k = 1}^{n}( {x_{k} - u} )^{2}}}.}}$

In another example, the pixel variance for the patch group is defined asthe variance of the current patch instead of the current patch group.

In some examples, the lookup table 500 can be used to replace the squareroot operation to obtain a pixel SD of a patch group such thatrespective computations can be simplified. In one example, the lookuptable 500 can include two columns: a first column 501 includes asequence of pixel SD values, and a second column 502 includes a sequenceof pixel variance values. In one example, the pixel SD values are in arange from 0 to 40, and labeled with numerals 521-527. The pixelvariance values are labeled with numerals 511-516. Each of the pixelvariance values 511-516 corresponds to a pixel SD value 521-526,respectively.

Particularly, each pixel variance value in column 502 is an average ofsquare values (variances) of two successive pixel SD values in column501. For example, the pixel variance value 511 equals an average ofsquare values of the pixel SD values 521 and 522. Thus, the pixelvariance value 511 is a value of the expression, (0̂2+1̂2)/2, which isshown at the place of the pixel variance value 511. Similarly, the pixelvariance value 512 equals an average of square values of the pixel SDvalues 522 and 523. Thus, the pixel variance value 512 is a value of theexpression, (1̂2+2̂2)/2, which is shown at the place of the pixel variancevalue 512. In a similar way, the pixel variance values 513-516 can becalculated. It is noted that the expressions for calculating averages ofsquare values of pixel SD values are shown in column 502 in FIG. 5 forexplanation purpose, and are not part of the lookup table 500. Instead,the results of each expression are contained in column 502 in variousexamples.

When a pixel variance for a patch group is obtained, by searching thelookup table 500, an estimated pixel SD value can be determined. Forexample, when a pixel variance is smaller than the pixel variance value511, the pixel SD value 521 can be used as the estimated pixel SD value.When a pixel variance is between a range between the pixel variances 511and 512, the pixel SD value 522 can be used as the estimated pixel SDvalue. As can be seen, when a value of a pixel variance falls betweentwo neighboring pixel variances, a pixel SD, a square value (variance)of which falls within a range between the two neighboring pixelvariance, can be determined to be the respective estimated pixel SD. Fora pixel variance that is larger than the last pixel variance 516, thelast pixel SD 527 can be used as the respective estimated pixel SD.

In one example, searching for an estimated pixel SD in the lookup table500 corresponding to an obtained pixel variance can be performed in thefollowing way. First, searching in the sequence of pixel variances inthe column 502 from the smallest variance 511 until a pixel variancethat is greater than the obtained pixel variance is found. Second, apixel SD in the sequence of the pixel SDs in column 501 corresponding tothe found pixel variance (e.g., at the same row in the lookup table 500)can be determined to be the estimated pixel SD.

It is noted that in various examples, structures of the lookup table 500can be different from what is shown in FIG. 5. For example, the sequencein column 501 or 502 can be ordered differently. Accordingly, specificsearching algorithms can be adapted to be suitable for the respectiveorders of the lookup table 500.

In some examples, the lookup table 500 can be constructed beforeperforming patch group denoising operations. In one example, a range anda precision of the pixel SDs in the lookup table 500 is first defined inorder to construct the lookup table 500. For example, when a compressionnoise model is formulated based on experiments, a range and a precisionof patch group pixel SDs can be determined accordingly. For example,based on statistic characteristics of reconstructed pictures,compression noise levels derived from the noise model can be limited toa certain range. Precisions of the compression noise levels can also beconsidered and determined. As a result, the compression noise model canhave a certain level of accuracy high enough to achieve a targetnon-local denoising processing performance and at the same timerespective processing cost can be controlled to be within a certainlevel. According to the compression noise model and respective range andprecision, the range and precision of patch group pixel SDs can bedefined.

For example, a range of patch group pixel SDs can be defined to be 0-10,0-40, or 0-100 for a pixel depth of 8 bits corresponding to differentreconstructed pictures in different examples. For example, a precisionof patch group pixel SDs can be 2 binary bits after a binary point.Accordingly, for patch group pixel SDs in a range of 0-40, there can be160 pixel SD values in column 501 in the lookup table 500. For example,the 160 pixel SD values in binary format can be 0.00, 0.01, 0.10, 0.11,1.00, 1.01, 1.10, 1.11, . . . , 101000.11.

Accordingly, in one example, a process for constructing of a patch grouppixel SD lookup table can include the following steps. First, a rangeand a precision of the pixel SDs can be determined. Second, a sequenceof pixel SD values can be determined according to the determined rangeand precision of the pixel SDs. Third, a sequence of pixel variances canbe determined according to the determined pixel SDs. For example, eachof the pixel variances can be an average of square values of twosuccessive pixel SDs.

FIG. 6 shows a second example patch group pixel SD lookup table 600according to an embodiment of the disclosure. The lookup table 600 caninclude a sequence of pixel variance values 611-616 in a column 602, forexample, from 0 to 1600, and a sequence of pixel SD values 621-626 in acolumn 601. Particularly, in each row of the lookup table 600, a pixelSD value takes a square root of a pixel variance value in the same row.For example, the pixel SD 623 takes a square root of the pixel variance613. While the square root operation expressions are shown in column601, the square root operation expressions are not part of the lookuptable 600. Results of the square root operations are contained in column601 of the lookup table 600 in various examples.

The lookup table 600 can be constructed in a way different from thelookup table 500. For example, in a process for creating the lookuptable 600, the pixel variances in column 602 can first be determined.For example, a range of the pixel SDs in column 601 can first be definedin a way similar to the FIG. 5 example. Accordingly, a range of therespective pixel variances can be defined. For example, if a range ofthe pixel SDs is from 0 to 40, the range of the corresponding pixelvariances can be from 0 to 1600. Based on the range of the pixelvariances, a sequence of pixel variances can be created. For example, inFIG. 6, a precision corresponding to integer pixel variance values areadopted, and the sequence in column 602 can include 1601 valuescorresponding to the range of 0-1600. Accordingly, when calculatingpixel variances of patch groups, the same integer precision can beadopted. In other words, results of calculating pixel variances of patchgroups can be rounded to integer values to be consistent with theto-be-used lookup table 600.

After the pixel variances in column 602 are determined, the pixel SDs incolumn 601 can be determined by performing square root operations torespective pixel variances. For example, the pixel SD 624 can bedetermined to be 1.7.

When searching the lookup table 600, a calculated pixel variance for apatch group can be used. When a pixel variance matching the calculatedpixel variance is found in column 602, a pixel SD in the same row can bedetermined to be the target pixel SD that can be used in a compressionnoise model for estimating a compression noise level.

Comparing the lookup table 600 with the lookup table 500, the lookuptable 500 is created based on pixel SDs, and pixel variances within arange are mapped to one pixel SDs. However, the lookup table 600 iscreated based on pixel variances, and one pixel variance is mapped withone pixel SD. The lookup table 600 can potentially have a larger sizethan the lookup table 500 corresponding to a same application.

Aspects of the disclosure provide example methods for simplifyingcalculation of weighting factors, for example, performed at S410 in FIG.4 example. At S410, a weighting factor of a reference patch with respectto a current patch in a patch group can be derived based on theexpression,

w _(i,j) =e ^(−(ASSE/Var)).

In the above expression, ASSE represents an average of a sum of squareerrors between pixel values in the reference patch and the currentpatch; noise Var represents a compression noise level in the patchgroup. Various methods can be employed to simplify calculation of theterm ASSE/Var.

In one example, a division operation 1/Var in the term ASSE/Var isreplaced with a division lookup table searching operation represented asLUT₁[Var]. Accordingly, the calculation of ASSE/Var is transformed asshown below,

ASSE/Var→ASSE×LUT₁[Var].

In the division lookup table searching operation, a search is performedin a first division lookup table that stores a sequence of values of1/Var each mapped with a value of Var.

In the above example, a range of values of Var can be large. Forexample, the range of Var values can be from 0 to 60,000. As a result, asize of the first division lookup table can be large. In order to reducethe lookup table size, in one example, a dynamic right shift operationis performed to Var to obtain K number of most significant bits (K-MSBs)of the Var where K is a preconfigured number. Then, a second divisionlookup table can be utilized to determine a value of 1/K-MSBs based onthe obtained K-MSBs.

For example, a Var value can be obtained from a compression noise model,and have a length of 10 binary bits represented as L. The number K isconfigured to be 4. Accordingly, in one example, the Var can be rightshifted by N_(x) bits until K most significant bits are left. In thisexample, a number of the right shifted bits equals L−K=10−4=6 bits. Asfor different patch groups, the noise variance Var can vary, theto-be-shifted bits number N_(x) can accordingly vary.

The second division lookup table can store a sequence of values of1/K−MSBs each corresponding to a K−MSB in another sequence in the seconddivision lookup table. Searching in the second division lookup table canbe based on a K−MSB. Compared with the first division lookup table, asize of the second division look up table is determined by the number ofK, thus being decreased.

Corresponding to the N_(x) bits right shifting operation performed onthe Var (the denominator of the term ASSE/Var), the numerator ASSE canbe similarly right shifted N_(x) bits to maintain a proper value of theASSE/Var. Accordingly, the transformation of calculation of ASSE/Varbased on the second division lookup table can be represented as follows,

ASSE/Var→(ASSE>>N _(x))×(LUT₂[Var>>N _(x)]),

where >> represents a right shift operation, and LUT₂[Var>>N_(x)]represents a lookup operation performed in the second lookup table witha noise variance, Var, right shifted by N_(x) bits.

In the above example, the values of 1/K−MSBs can be small because avalue of a K−MSB can be a large integer. To reduce cost associated withstoring and calculating with the small values of 1/K−MSBs, in oneexample, the second division lookup table is modified resulting a thirddivision lookup table. Specifically, the third division lookup table canstore values of 2̂M/K−MSBs where M is a preconfigured integer number. Inthis way, corresponding to a K−MSB, an exaggerated value that is 2̂Mtimes of the 1/K−MSB can be located. The exaggerated value, 2̂M/K−MSBs,can be more suitable for storing or manipulating compared with the valueof 1/K−MSB, leading to higher processing efficiency.

Corresponding to the exaggeration operation in the third division lookuptable, a resulting value can be right shifted for M bits to obtain aproper value of a respective ASSE/Var. Accordingly, the calculation ofASSE/Var with the third division lookup table can be represented asfollows,

ASSE/Var→((ASSE>>N _(x))×(LUT₃[Var>>N _(d)]))>>M,

where LUT₃[Var>>N_(x)] represents a lookup operation performed in thethird lookup table with a noise variance, Var, right shifted by N_(x)bits.

In some examples, rounding offsets can be applied to further improveaccuracy of calculating the term ASSE/Var. For example, beforeperforming a right shifting to a noise variance Var, a rounding offsetcan be added to Var. For example, a value of the rounding offset cantake (1<<N_(x))/2 where <<represents a left shift operation. In variousexamples, a value of the rounding offset can be in a range from 0 to(1<<N_(x))−1. Similarly, a rounding offset can be added to the value ofASSE or ((ASSE>>N_(x))×(LUT₃[Var>>N_(x)])) before performing arespective right shifting operation. In addition, rounding offsets canbe used to one or multiple of the following terms: ASSE, Var, or((ASSE>>N_(x))×(LUT₃[Var>>N_(x)])) for one denoising process. Differentchoices regarding which term(s) to apply rounding offsets can beselected for difference video sequences, reconstructed pictures, orpatch groups in various examples.

As an example, FIG. 7 shows a process 700 for calculating a termASSE/Var using the third division lookup table according to anembodiment of the disclosure. The process 700 can start from S701, andproceed to S710.

At S710, the Var can be right shifted by N_(x) bits to locate K numberof most significant bits (K−MSBs).

At S720, a value of 2̂M/K−MSBs can be determined by searching the thirddivision lookup table based on the located K−MSBs.

At S730, the ASSE can be right shifted by N_(x) bits.

At S740, the right-shifted ASSE can be multiplied with the determinedvalue of 2̂M/K−MSBs to obtain a product.

At S750, the product can be right shifted by M bits to obtain the targetASSE/Var value. The process 700 can then proceed to S799, and terminateat S799.

Aspects of the disclosure provide example methods for simplifyingnormalization operations performed on accumulated pixel values, forexample, at S450 in FIG. 4 example. At S450, accumulated pixel values inan accumulation picture are normalized to obtain a filtered picture. Forexample, an accumulated pixel, represented as X_(A), can includemultiple portions of pixel values, and each of the portions correspondsto an original pixel value of either a current patch or a referencepatch multiplied by a gain. Each gain corresponds to a weighting factor,or a product of one or more weighting factors. Accordingly, anaccumulated pixel can be divided by a sum of those gains (represented asS_(G)) to obtain a normalized pixel value, X_(A)/S_(G). The divisionoperation in X_(A)/S_(G) can be simplified to increase efficiency ofpatch group processing.

In one example, calculation of the term X_(A)/S_(G) can be performed ina way similar to calculation of ASSE/Var with lookup tables similar tothe first, second, or third division lookup table. As an example, FIG. 8shows a process 800 for calculating X_(A)/S_(G) according to anembodiment of the disclosure. The process 800 can start with S801, andproceed to S810.

At S810, the sum of the gains S_(G) can be right shifted by N_(x) bitsto locate K number of most significant bits (K−MSBs).

At S820, a value of 2̂M/K−MSBs can be determined by searching a lookuptable similar to the third division lookup table described above basedon the located K−MSBs.

At S830, the X_(A) can be right shifted by N_(x) bits.

At S840, the right-shifted X_(A) can be multiplied with the determinedvalue of 2̂M/K−MSBs to obtain a product.

At S850, the product can be right shifted by M bits to obtain anormalized pixel value. The process 800 can then proceed to S899, andterminate at S899.

In various examples, rounding offsets can be applied to improve accuracyof calculation of the term X_(A)/S_(G). As an example, FIG. 9 shows aprocess 900 with rounding offset operations according to an embodimentof the disclosure. The process 900 can start at S901, and proceed toS910.

At S910, the term X_(A)/S_(G) can first be calculated without addingrounding offsets, for example, using a lookup table similar to thesecond or third division lookup table described herein.

At S920, the term X_(A)/S_(G) can be calculated with rounding offsetsadded using the same lookup table as in the first step. Specifically,the rounding offsets can be added to one or multiple of the followingterms: X_(A), S_(G), and ((X_(A)>>N_(x))×(LUT₃[S_(G)>>N_(x)])). Forexample, the rounding offsets can be added to X_(A) or S_(G), or both.

At S930, results of the first and second steps are averaged, and theaveraged value is used as a target value of the term X_(A)/S_(G). Theprocess 900 can then proceed to S999, and terminate at S999.

While aspects of the present disclosure have been described inconjunction with the specific embodiments thereof that are proposed asexamples, alternatives, modifications, and variations to the examplesmay be made. Accordingly, embodiments as set forth herein are intendedto be illustrative and not limiting. There are changes that may be madewithout departing from the scope of the claims set forth below.

What is claimed is:
 1. A method, comprising: receiving reconstructedvideo data corresponding to a picture; dividing the picture into currentpatches; forming patch groups each including a current patch and anumber of reference patches that are similar to the current patch;determining a noise level for each of the patch groups, including:calculating a pixel variance of pixel values for a respective patchgroup, determining a pixel standard deviation (SD) of the respectivepatch group according to the calculated pixel variance for therespective patch group by searching in a first lookup table thatindicates mapping relationship between patch group pixel SDs and patchgroup pixel variances, and calculating a noise level for the respectivepatch group based on a compression noise model that is a function of thepixel SD of the respective patch group; and denoising the patch groupswith a non-local denoising technology to create a filtered picture basedon the determined compression noise level of each of the patch groups.2. The method of claim 1, wherein the first lookup table includes afirst sequence of patch group pixel SDs, and a second sequence of patchgroup pixel variances, wherein an average value between square values ofeach pair of two successive patch group SDs forms the second sequence ofpatch group variances.
 3. The method of claim 2, wherein the searchingin the first lookup table includes: searching for a range defined by twosuccessive average values in the second sequence that includes the pixelvariance for the respective patch group; and determining a patch groupSD in the first sequence to be the pixel SD of the respective patchgroup, wherein a square value of the determined patch group SD isincluded in the same range defined by the two successive average valuesin the second sequence.
 4. The method of claim 2, further comprising:constructing the first lookup table, including, determining a range ofthe first sequence of patch group SDs, determining a precision of thefirst sequence of patch group SDs, and determining the first sequence ofpatch group SDs according to the determined range and precision.
 5. Themethod of claim 1, wherein the first lookup table includes a firstsequence of patch group SDs, and a second sequence of patch groupvariances, wherein each of the first sequence of patch group SDs is anSD of one of the second sequence of patch group variances.
 6. The methodof claim 5, wherein searching in the first lookup table includes:finding a patch group variance in the second sequence that is equal tothe pixel variance for the respective patch group; and determining oneof the first sequence of patch group SDs corresponding to the foundpatch group variance of the second sequence to be the pixel SD of therespective patch group.
 7. The method of claim 1, wherein the non-localdenoising technology is one of: a non-local means (NLM) denoisingtechnology, a block matching and 3D filtering (BM3D) denoisingtechnology, or a low-rank approximation (LRA) denoising technology. 8.The method of claim 1, wherein denoising the patch groups with thenon-local denoising technology to create the filtered picture based onthe determined compression noise level of each of the patch groupsincludes: calculating an ASSE/Var value to be used for calculating aweighting factor of a reference patch with respect to a current patch,ASSE being an average of a sum of square errors between the referencepatch and the current patch, Var being a noise variance that is a squareof a noise level of a patch group that includes the reference patch andthe current patch, wherein calculating the ASSE/Var value includes,right shifting the Var by N_(x) bits to locate K number of mostsignificant bits (K-MSBs), K being a preconfigured number, determining avalue of 2̂M/K−MSBs by searching a second lookup table based on thelocated K−MSBs, M being a preconfigured number, right shifting the ASSEby N_(x) bits, multiplying the right-shifted ASSE with the determinedvalue of 2̂M/K−MSBs to obtain a product, and right shifting the productby M bits to obtain the ASSE/Var value.
 9. The method of claim 10,wherein the second lookup table indicates mapping relationship between asequence of values of 2̂M/K−MSBs and a sequence of K−MSBs each having Knumber of bits.
 10. The method of claim 10, further comprising one of:adding a rounding offset to the Var before right shifting the Var byN_(x) bits; adding a rounding offset to the ASSE before right shiftingthe ASSE by N_(x) bits; and adding a rounding offset to the productbefore right shifting the product by M bits.
 11. The method of claim 1,wherein denoising the patch groups with the non-local denoisingtechnology to create the filtered picture based on the determinedcompression noise level of each of the patch groups includes:calculating a normalized pixel value corresponding to an accumulatedpixel value X_(A) that includes multiple portions of pixel values, eachof the portions corresponding to an original pixel value of the dividedpicture multiplied by a gain, wherein calculating the normalized pixelvalue includes, (a) right shifting a sum of the gains, S_(G),corresponding to the accumulated pixel value X_(A) by N_(x) bits tolocate K number of most significant bits (K−MSBs), K being apreconfigured number, (b) determining a value of 2̂M/K−MSBs by searchinga third lookup table based on the located K−MSBs, M being apreconfigured number, (c) right shifting the X_(A) by N_(x) bits, (d)multiplying the right-shifted X_(A) with the determined value of2̂M/K−MSBs to obtain a product, and (e) right shifting the product by Mbits to obtain a first normalized pixel value.
 12. The method of claim11, wherein the third lookup table indicates mapping relationshipbetween a sequence of values of 2̂M/K−MSBs and a sequence of K−MSBs eachhaving K number of bits.
 13. The method of claim 11, wherein calculatingthe normalized pixel value further includes: performing steps of (a)-(e)with a rounding offset operation to obtain a second normalized pixelvalue, the rounding offset operation including at least one of, adding arounding offset to the S_(G) before right shifting the S_(G) by N_(x)bits, adding a rounding offset to the X_(A) before right shifting theX_(A) by N_(x) bits, or adding a rounding offset to the product beforeright shifting the product by M bits; and averaging the first and secondnormalized pixel values to obtain the calculated normalized pixel value.14. A non-local adaptive loop filter, comprising circuitry configuredto: receive reconstructed video data corresponding to a picture; dividethe picture into current patches; form patch groups each including acurrent patch and a number of reference patches that are similar to thecurrent patch; determine a noise level for each of the patch groups,wherein determining the noise level includes: calculating a pixelvariance of pixel values for a respective patch group, determining apixel standard deviation (SD) of the respective patch group according tothe calculated pixel variance for the respective patch group bysearching in a first lookup table that indicates mapping relationshipbetween patch group pixel SDs and patch group pixel variances, andcalculating a noise level for the respective patch group based on acompression noise model that is a function of the pixel SD of therespective patch group; and denoise the patch groups with a non-localdenoising technology to create a filtered picture based on thedetermined compression noise level of each of the patch groups.
 15. Thenon-local adaptive loop filter of claim 14, wherein the first lookuptable includes a first sequence of patch group pixel SDs, and a secondsequence of patch group pixel variances, wherein an average valuebetween square values of each pair of two successive patch group SDsforms the second sequence of patch group variances.
 16. The non-localadaptive loop filter of claim 14, wherein the first lookup tableincludes a first sequence of patch group SDs, and a second sequence ofpatch group variances, wherein each of the first sequence of patch groupSDs is an SD of one of the second sequence of patch group variances. 17.The non-local adaptive loop filter of claim 14, wherein the circuitry isfurther configured to: calculate an ASSE/Var value to be used forcalculating a weighting factor of a reference patch with respect to acurrent patch, ASSE being an average of a sum of square errors betweenthe reference patch and the current patch, Var being a noise variancethat is a square of a noise level of a patch group that includes thereference patch and the current patch, wherein calculating the ASSE/Varvalue includes, right shifting the Var by N_(x) bits to locate K numberof most significant bits (K-MSBs), K being a preconfigured number,determining a value of 2̂M/K−MSBs by searching a second lookup tablebased on the located K−MSBs, M being a preconfigured number, rightshifting the ASSE by N_(x) bits, multiplying the right-shifted ASSE withthe determined value of 2̂M/K−MSBs to obtain a product, and rightshifting the product by M bits to obtain the ASSE/Var value.
 18. Thenon-local adaptive loop filter of claim 14, wherein the circuitry isfurther configured to: calculate a normalized pixel value correspondingto an accumulated pixel value X_(A) that includes multiple portions ofpixel values, each of the portions corresponding to an original pixelvalue of the divided picture multiplied by a gain, wherein calculatingthe normalized pixel value includes, (a) right shifting a sum of thegains, S_(G), corresponding to the accumulated pixel value X_(A) byN_(x) bits to locate K number of most significant bits (K−MSBs), K beinga preconfigured number, (b) determining a value of 2̂M/K−MSBs bysearching a third lookup table based on the located K−MSBs, M being apreconfigured number, (c) right shifting the X_(A) by N_(x) bits, (d)multiplying the right-shifted X_(A) with the determined value of2̂M/K−MSBs to obtain a product, and (e) right shifting the product by Mbits to obtain a first normalized pixel value.
 19. The non-localadaptive loop filter of claim 18, wherein the circuitry is furtherconfigured to: perform steps of (a)-(e) with a rounding offset operationto obtain a second normalized pixel value, the rounding offset operationincluding at least one of, adding a rounding offset to the S_(G) beforeright shifting the S_(G) by Nx bits, adding a rounding offset to theX_(A) before right shifting the X_(A) by Nx bits, or adding a roundingoffset to the product before right shifting the product by M bits; andaverage the first and second normalized pixel values to obtain thecalculated normalized pixel value.
 20. A non-volatile computer-readablemedium storing instructions that, when executed by a processor, causethe processor to perform a method, the method comprising: receivingreconstructed video data corresponding to a picture; dividing thepicture into current patches; forming patch groups each including acurrent patch and a number of reference patches that are similar to thecurrent patch; determining a noise level for each of the patch groups,including: calculating a pixel variance of pixel values for a respectivepatch group, determining a pixel standard deviation (SD) of therespective patch group according to the calculated pixel variance forthe respective patch group by searching in a first lookup table thatindicates mapping relationship between patch group pixel SDs and patchgroup pixel variances, and calculating a noise level for the respectivepatch group based on a compression noise model that is a function of thepixel SD of the respective patch group; and denoising the patch groupswith a non-local denoising technology to create a filtered picture basedon the determined compression noise level of each of the patch groups.